One of the most challenging problems in computer graphics today is rendering visually compelling three-dimensional (3D) environments. For real-world environments, a key problem is to recreate the complex interaction between geometric and photometric properties.
Image-based rendering (IBR) attempts to address this problem by capturing images of the environment and then creating novel views by re-sampling the images which already contain the geometric and photometric properties. One example of an application that uses IBR techniques is an interactive walkthrough. An interactive walkthrough is a computer graphics application where an observer moves within a virtual environment. For an interactive walkthrough application to be effective, rendering cannot be limited to coincide with the captured viewpoints or captured paths, but instead requires image reconstruction from arbitrary viewpoints or along arbitrary viewpoint paths.
The main tasks for an IBR system are: (1) to acquire a sampling of calibrated reference images; (2) to warp reference images so that their centers-of-projection coincide with the location of the virtual observer; and (3) to combine the warped images from multiple reference images to form a novel image at the location of the virtual observer. The first and third of these tasks have been well studied in recent years. However, the second task (to warp reference images so that their centers-of-projection coincide with the location of the virtual observer) has still not been addressed adequately.
Conventional approaches for performing the second task require computing either pixel correspondences between the images or an accurate 3D scene geometry. However, it is difficult to achieve accurate pixel correspondence, and the 3D scene geometry is often not available and even more difficult to extract from a set of images. As a result, most IBR systems have produced novel images with noticeable blurring and ghosting artifacts, or they have been demonstrated only with distant or synthetic scenes.
Thus, there exists a need for techniques that overcome the above-mentioned drawbacks by providing improved techniques for warping images captured in real-world environments so as to enable improved arbitrary viewpoint rendering in such applications as interactive walkthroughs.